Papers with semantic sensitivity
Beyond Length: Context-Aware Expansion and Independence as Developmentally Sensitive Evaluation in Child Utterances (2026.eacl-long)
Copied to clipboard
| Challenge: | Common proxies such as Mean Length of Utterance (MLU), lexical diversity (vocd-D), and readability indices are dominated by length and ignore conversational context, missing aspects of response quality such as reasoning depth, topic maintenance, and discourse planning. |
| Approach: | They propose a framework that classifies the Previous Adult Utterance Type and scores the child’s response along two axes: Expansion (contextual elaboration and inferential depth) and Independence (the child’ s contribution to advancing the discourse). |
| Outcome: | The proposed framework assesses the child's response along two axes: Expansion (contextual elaboration and inferential depth) and Independence (the child’s contribution to advancing the discourse). |
SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate open domain dialogues have a one-to-many problem . existing approaches lack commonsense reasoning biases and perform poorly in domain-specific scenarios. |
| Approach: | They propose a framework that leverages both a small, specialised model and LLMs for the evaluation of open-domain dialogues. |
| Outcome: | The proposed framework achieves state-of-the-art performance in both classification and evaluation tasks and exhibits better correlation with human judgements. |